1. Field of the Invention
This invention relates to networks for processing information and more particularly, to neural networks implemented in semiconductor device technology.
2. Description of the Prior Art
Much interest has been generated recently in the field of neural networks for processing information. A neural network is an electronic network which in some fashion mimicks the structure and function of the neurons found in living organisms. The ultimate goal is to construct an artificial system that can display all or some of the powerful functions of biological systems.
The scope of the work that has been done in the area of neural networks is quite extensive. In one prior art system, key features of the organic neuronal system are used in designing an electronic network to achieve a better understanding of these same biological features, see Hopfield, Proceedings National Academy Science USA 79,2554 (1982), while another system tries to imitate the functioning of the neurons in intricate detail, see El-Leithy, et al., IEEE First International Conference on Neural Networks (1987). Some networks have been modeled and/or constructed in order to solve very specific problems, such as speech generation, see Sejnowski, et al., John Hopkins University Technical Report JHU/EECS-86/01, 1986, or vision, see Ballard, et al., Nature 306, 21 (1983). Other networks seek to be more general in scope, hoping to define solutions to a whole class of problems, see Hopfield, et al., Biological Cybernetics 52, 141 (1985).
The Hopfield network, which is described in detail in U.S. Pat. No. 4,660,166, is a complex network of interconnections of non-linear switches that mimick the behavior of a spin glass. The Hopfield network is a matrix of output and input conductors where the output conductors form a plurality of "neurons". Each neuron of the network consists of a series of inputs and a simple threshold switch, with each input being multiplied by a parameter which is either learned or programmed into the network. The input products are then summed together and if the sum exceeds a particular threshold, the output of the neuron is set to the "on" state, otherwise the output will be in the "off" state. The output of each neuron is connected to the input of every neuron in the network thereby forming a totally interconnected network. The essential "memories" built into the network are contained in the multiplicative parameters applied to the neuronal inputs. The multiplicative parameters are provided by resistors coupled to each of the threshold switches.
The Hopfield type of network has been shown to display a large number of desirable qualities which are at least reminiscent of the functioning of biological neural networks. For example, the network can solve pattern recognition problems in a parallel mode. In addition, the network is capable of finding the closest match pattern in memory to an input pattern even when many of the input bits are in error or missing. The network is considered to be a content addressable memory. The memories are distributed over the network and are not contained in any one physical portion of the network. Therefore, the network can successfully operate when a portion of the network is erased or altered.
This type of network does have several limitations. The network will always return a pattern in response to the specified input which may not be the pattern desired. Furthermore, it does not have the built in capability to decide if the match is too far off from the input that no result should be returned.
While the Hopfield type network has been shown to work in computer simulations, difficulty has arisen in implementing the network in hardware devices. More particularly, it would not be practical to make a very large network in which the memory elements are embodied by resistors and the active elements are multitransistor circuits. U.S. Pat. No. 4,760,437 discloses one implementation of a Hopfield neural network by utilizing photoconductors as the memory elements with the desired impedance being established by controlling the level of light incident thereon. Another prior art example of the fabrication of a real network is shown by Graf, et al., see Proceedings of IEEE First International Conference on Neural Networks, 461 (1987), in which there is disclosed a CMOS associative memory chip for implementing the Hopfield matrix of resistive coupling elements interconnecting an array of transistor amplifiers. In contrast with the Hopfield network the data stored by the network is represented locally in the matrix thereby reducing the number of required interconnections.